Field of the Invention
The present invention relates to an image processing apparatus and an image processing method.
Description of the Related Art
With increased digitalization of image production workflow in recent years, it has become possible to easily generate a combined image in which a captured image generated by capturing using an image capturing apparatus and a computer graphic image (hereinafter, “CG image”) are combined.
Generation of a combined image is generally performed in post-production that is a step after a capturing step (step of performing capturing). Therefore, if there is a non-editable portion in a captured image at the time of generating a combined image, rework in the capturing step may occur.
Thus, a method is often employed in which a combining apparatus that generates (simply generates) a combined image is brought to a capturing location to generate and check a combined image at the capturing location. As the combining apparatus, a PC can be used, for example.
With such a method, a combined image can be checked (simply checked) in the capturing step, and the number of reworks in the capturing step can be reduced.
There are cases where the display brightness (brightness on screen) of a captured image differs greatly from the display brightness of a CG image.
In post-production, the display brightness of at least one of the captured image and the CG image is finely adjusted by an editor, so that the display brightness of the captured image is fit for the display brightness of the CG image.
However, since an editor is not always present at a capturing location, it is preferable that the display brightness of an image be automatically adjusted (simply adjusted) at a capturing location.
A technique of automatically adjusting the display brightness of an image is disclosed in Japanese Patent Application Laid-open No. 2005-142680, for example. In the technique disclosed in Japanese Patent Application Laid-open No. 2005-142680, the display brightness of each image is adjusted based on the average gradation value of one image, upon combining a plurality of images.
Along with the improvement in light-receiving performance of image capturing apparatuses in recent years, captured images having a wider dynamic range are being generated. As a data format for handling a captured image with a wider dynamic range, a data format having display characteristics (logarithmic characteristics) such that the display brightness changes logarithmically with respect to a change in the gradation value has been proposed. For example, at film production site, Cineon Log defined based on the characteristics of a film with a wide dynamic range is used as a data format.
On the other hand, a CG image generally has a narrow dynamic range. For example, BMP or JPEG is used as a data format for a CG image that is a still image, and JPEG 2000 is used as a data format of a CG image that is a video. The data formats cannot handle an image with a wide dynamic range, since the number of bits of a gradation value is 8 bits.
Therefore, there are cases where the dynamic range of a captured image and a dynamic range of a CG image greatly differ.
In the case where the dynamic ranges of a captured image and a CG image are the same, a suitable adjustment result (result of adjustment in the display brightness of an image) can be obtained by using the technique in Japanese Patent Application Laid-open No. 2005-142680.
However, in the case where the dynamic ranges of a captured image and a CG image are different, a suitable adjustment result cannot be obtained, even if the technique in Japanese Patent Application Laid-open No. 2005-142680 is used. For example, in the case of displaying a combined image in which an HDR image and an LDR image are combined, the display brightness of the HDR image cannot be made fit for the display brightness of the LDR image, even if the technique in Japanese Patent Application Laid-open No. 2005-142680 is used. An HDR image is an image with a wide dynamic range, e.g., a captured image. An LDR image is an image with a narrow dynamic range, e.g., a CG image.
An example of displaying a combined image in which an HDR image and an LDR image are combined will be described below.
As shown in FIG. 9, the LDR image has display characteristics (linear characteristics) such that the display brightness changes linearly with respect to a change in the gradation value (LDR gradation value).
On the other hand, as shown in FIG. 10, the HDR image has display characteristics (logarithmic characteristics) such that the display brightness changes logarithmically with respect to a change in the gradation value (HDR gradation value). The dynamic range of the HDR image is wider than that of the LDR image. The human vision is sensitive to a dark area, and it is preferable to allocate a narrow gradation range (range of gradation value) with respect to a range of low display brightness. Therefore, the logarithmic characteristics are frequently used as the display characteristics of an HDR image.
In FIGS. 9 and 10, a gradation value corresponding to a light intensity value is used as the gradation value, in order to clarify the difference of dynamic ranges of the HDR image and the LDR image. The light intensity value is a value converted from a gradation value according to the reverse characteristics of the gradation characteristics that are a change in the gradation value with respect to a change in the light intensity value. In the case where an image is a captured image, the light intensity value is a value representing the brightness of light (brightness of a captured scene) received by an image sensor (CMOS sensor or the like) of an image capturing apparatus. In the case where the gradation characteristics are gamma characteristics, a value converted from a gradation value according to reverse gamma characteristics coincides with the light intensity value.
As shown in FIGS. 9 and 10, the display brightness corresponding to the HDR gradation value differs from the display brightness corresponding to the LDR gradation value, even if the HDR gradation value and the LDR gradation value are equal. Therefore, a suitable adjustment result cannot be obtained, even if the technique in Japanese Patent Application Laid-open No. 2005-142680 is used. For example, the display brightness of the HDR image cannot be caused to coincide with the display brightness of the LDR image, even if the gradation value of the HDR image or the LDR image is adjusted such that the average gradation value of the HDR image coincides with the average gradation value of the LDR image.